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Very fev; planned domestic efforts in this country have surpassed 

those made by metropolitan school systems during the last decade. Massive 

amounts of time, money, and energy have been spent on the creation and 

introdufction of innovative programs into inner-city schools to offset 

2 

the poor academic performance of children in them. These efforts have 

been accompanied by the literally thousands of studies, massive in their 

own right, designed to measure the effects of these innovative programs 

and by a growing research literature devoted to ai'ticies and books on 

3 

how to assess the effectiveness of educational innovations, 

V/hile the literature and many of the studies contained in it discuss 
at lehgth the problems and procedures of evaluations, surprisingly little 
attention is given to how data collected during evalueitions may be used 
to increase the understanding of the educative process and to shed light 
on social and psychological theories related to education. The plain 
fact is that, since this has not been of central concern to evaluators, 
theory related to education has benefitted marginally from these thou- 

4 

sands of studies. 



*A paper to be presented at the 1972 annual meeting of the 
American Educational Research Association in Chicago, April 1972. 
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One reason for this sad state of affairs is that the conditions 
under which evaluation data are collected are often inadequate for 
research purposes and this renders them useless as the basis for 
developing or confirming theories _ and hypotheses, ?, second reason is 
that, even when collected under acceptable conditions, data from one 
study cannot be related in any systematic fashion to data gathered in ' 
another, so that no benefit can be derived by accumulating results, 

A third reason is that investigators fail to treat evaluation studies 
as opportunities to also collect evidenc in actual field settings to 
explore meaningful theoretical problems, it is with this objective in 
mind that the research reported in this paper was undertaken. 



I 

The Social Sc i ence Problem to be Explorf !.^ 

During an evaluation study, data were collected ■.:o tea:; the useful- 
ness of a theoretical model, which includes privary :.oudii:j.oris necessary 
for the implementation of proposed organizational changes i:-.- schools and 
oth'.-r eetf ags such as businesses, factories, boor.italv., Implemen- 

tacx>y.i may be d>^^ined as a process leading to the ch.'.oged role performance 

r 

of organisational mombers after the introduction of an innovation,^ 
Eval.uation studies, such as this one, provide excellent oppsortunities to 
explore this theoretical problem in school settings. 

During the 1970—71 academic year, an assessment of educational 
programs funded under Title I of ESEA in a New York City Schoo^ District 
was conduohed. The district was fairly representative of inncr-city 
areas: housing varied from decrepit to acceptable; most families were 

lower class and non-white; and many were on public assistance. 
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The primary objective of the evaluation study was to measvire the 
effects of these programs (10 in all) on the schools and the students 
within them. Five of the programs in the District's 21 schools were 
designed specifically to offset the poor academic performance of students 
by providing greater individual instruction through the use of para- 
professional assistance. These five programs varied in size: the 

Preprimary Program involved nearly 45 paraprofessionals in early child- 
hood and kindergarten classrooms in 10 schools; the Early Primary Pro- 
gram had some 150 paraprofessionals located in first and second grade 
classes in 11 schools; the Late Primary Frogrc.m involved 35 parapro- 
fessionals in third grade classrooms in 11 schools; the Intermediate 
Program contained 75 paraprofessionals in fourth, fifth, end 'sixth 

grade classes in 8 schools; and 75 paraprofessionals v;ere assigned to 

( 

classrooms in the four junior high schools.^ 

/ 

The paraprofessionals, themselves, v;ere neerly all non-white- women, 

30 to 50 years of age. i)ll had- at least high school diplomas, supple- 
mented by District-sponsored training. Many were enrolled in community 
college progrcgjjs, with some working toward their bachelor's degree. 

The five programs were treated as organizational innovations because 
they involved the introduction of a now position in schools at the class- 
room level, which resulted in a new set of role relationships. J^lthough 
similar in many respects, each program was treated as a discrete innovation 
because the role of the paraprofessional differed by grade level. For 
example, the Preprimary paraprofessionals were expected to spend much of 
their time on activities that v;ould promote child socialization and much 
less time on developing reading skills; the reverse was true for the 
paraprofessiohals in the Jmior High Program. 

/ 

/ 
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Pertinent Social Science Literature 
The study of planned chcnge in the prst hes emphasized severe! 

7 ' p 9 

ereas. In their writings, Cerlson , Lionberger®, and Rogers under- 
score the interest many investigators have in explaining the rates at 
v;hich simple, technological innovations are diffused and adopted among 
individuals ~ e.g», hybrid seed among farmers, hand tools among 
villagers, tranquilizers among doctors, end audio-visual aids among 
school superintendents (representing school systems). Some of the 
major questions examined by researchers have been: Tliat are the stages 

that individuals go through in the adoption process? Hov; do charac- 
teristics of the innovations influence their rates of diffusion? IThat 

t 

jiersonel end social attributes' distinguish early from late adopters? 

The work of Dennis, Benne, and Chin^® and Lippitt, T'estleyi and 
Watson^^ reflect an area of interest of still other investigators, one 
more clearly associated with schools as organizations: the initiation 

of organizational changes — the process whereby innovations are intro- 
duced and adopted in organizations’^'^. Here a majw^pneern has been to 

determine to what extent the use of tactics such as hiring change agents, 

'* '■ ...... 

permitting subordinates- to participate in decision-making, or both, leads 
to successful initiation of changes. 

h recent appraisal of the planned change literature by Gross, 
Giacquinta, and Bernstein^^, hov;ever, reveals that the problem of imple- 
menting organizational innovations, as caiipared to the problem of ini- 
tietion, has received very little systematic empirical study. This study 
notes that much of the discussion of implementation usually relates to the 
potential resistance of members tov;ard changes v;hich are introduced, end 
to the conditions necessary for overcoming this resistance. It calls 

4 
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into question a nvcuber of the eisaur.iptions underlying this reasoning 
about resistance and points to the critical need for studying imple- 
mentation es a. process scpcrete from initiation. 

The theoretical model describing • the factors that are at the - 
heart of implementation and emerging from this recent work, serves as 
the basis for this present investigation in schools.. Gross and his 
associates propose^ that the implementation of organizational innovations 
having once been initiated requires the presence of five conditions, 
each of which is necessary but alone insufficient. These antecedent 
conditions of implemehtation are that* 1) organizational members who 
must carry out an innovation clearly understand the new expectations; 

2) they have the abilities necessary for the nev; role performance; 

3) they are willing to raake the required efforts to behave in accord 
with the innovation; 4) the materials and resources required by the 
innovation are available; 5) the organization is compatible end 
supportive of the innovation 

According to this model, the degree to which innovations are 
implemented is a function of the extent to which these conditiojis in 
combination are present. Thus, programs or schools that rank higher 
on these conditions should rank higher on implementation than those 
that rank lov;er on these conditions. Moreover, programs or schools 
(containing programs) v;here the five conditions ere highly present 
should demonstrate high levels of implementation. The evaluation 
study, from which this investigation emanates, permitted the collec- 
tion of data that could be used to test the usefulness of this model 
in explaining variations in level of implementation among five inno- 
vative paraprofessional programs and among several schools within tv;o 
^ of these five programs, 

er|c 
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Sample " ' - . 

Since the number of perc professions Is in esch program varied, 
a proportionate, stratified random sample was drown. From the total 
of approximately 390 par apr of essionals working in 'these programs, 

127 v.'ere selected, /'if ter the sample of par apr of essionals was chosen, 
the teacher with whom each worked was identified. The original total 
sample for the study, ^ therefore, was 254 individuals, lialf of v;hom 
were par apr of essionals and the other half teachers. A number of 
administrators in each program were also selected and used primarily 

’ I 

for purposes of research beyond the scepe of this study. 

'i 

■ ■ V . , 

D ata Collection . ^ ' 

Two instruments v;ere employed in the collection of data. The 
purposjs of the Role Definition Questionnaire (RDQ) was two~f.old: 

First, it provided evidence about the role definition of' the parapro* 
fessional to be used in assessing implementation; second, it provided 
information for one of the independent variables, clarity of role. The 
instrument is comprised of 56 items that reflect activities often related 
to the role of .the pcraprofessional. . ; ’ 

Five trained research assistants administered the RDQ during March, 
1971 to the 127 paraprofessionals in our sample. Employed for the on- • 
going evaluation of the paraprofessional programs, the Assistants, in 

r* I 

most cases, had previous contact with the subjects. The questionnaire, 
accompanied by thorough instructions, was personally delivered to each 
subject. Arrangements were made for the questionnaires to be collected 
by the research assistants', ^ In the few cases where questionnaires were 
not completed, subjects were asked to return them by mail. The personal 
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contact by the research assistants contributed to the high rate of 
returns, v;hich ranged from 80 to 100 per cent (see Table 1) , /• 

The second instrument constructed for the present study was the 
Implementation Questionnaire (II'IQ). The purpose of the instrument was 
to gather demographic information, evidence on the degree of imple- . 
mentation, and data on four of the independent variables: ability, 

willingness, availability of resources and materials, and organizational 
compatibility. The instrument v?as administered in May, 1971 to the 
sample of paraprofessionals and teachers as v;ell as to some adminis- 
trators, 7 s mentioned earlier, the expected sample size v.»as 254, 
However, some attrition of paraprofessionals occurred betv;een the ad- 
ministration of the RDQ and the IMQ due primarily to reassignment. 

Since the paraprofessionals and teachers v?ere selected together, the 
attrition rate in the original sample v:as doubled. The same research 
assistants administered the IMQ in the manner described above. Returns 
for each program ranged from 74 to 100 per cent (see Table 1), 

Consis te ncy of Respondents : To obtain a measure of the respondents* 

• ♦ 

consistency, ten of the original 56 items- on the RDQ v;ere randomly 

selected, reworded, end included at the end of the questionnaire, 

1 

Consistency was considered to reflect the seriousness with which the 

\ 

paraprofessionals viewed their responses to the RDQ. Correlations 
betv;een responses to the original items and the paraphrased versions 
ranged from ,10 to , 53 and are listed in Table 2, Till but one of the 
ten correlations were significant, although they were not as high 
as had been anticipated^ , ' 
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Several reasons may. account fot the rather lov; coirelations, 
but these reasons do not reduce the confidence in ,the accuracy of 
paraprofessionel responses to the questionnaire. First, to guard 
against a response set, the direction of the scale .for the ten re- 

, f 

vjorded items v’as reversed,' In addition, the respondents v?ere asiced 
to record rather than , circle their c]lioices. In spite, of careful 
instructions, many paraprofessionals could have simply continued 
using the’ older response' scale thereby lowering the correlations. 
Second, although the consistency items v;ere carefully paraphrased to 
maintain the content of the original ones, the rev;ording of the items 
could have introduced dimensions not present in the originals, thereby 
influencing the responses to them and lov;ering the correlations. 

Third,' since the consistency -i terns v;ere attached to the end of a 
5S~item questionnaire, fatigue may have taken its toll resulting in 
less careful selections. 

Inspection"* of the questionnaires themselves and the parapro- 
fessionals* comments to the assistants who gathered them revealed 

A, 

that a. great deal of concern- and effort v;ere given to the answers. 
Nonetheless, it is still possible that the low correlations reflect 
inconsis-fcencies of respondents due to several reasons: the level of 
reading comprehension by the pera'profoss'ionalS may have been inade- 
quate; the ratings may ha^e required too fine a discrimination by 
the paraprofessionals ; and, the effort of the' paraprofessionals may 

have| been minimal resulting in haphazard responses, 

■ 0 • ■ . • 
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Measurcrnent- of Vrriebles • . . ' •>, i 

The utility of this theoretical model v;es tested with three Sets 

of enslysis. The first set involved comparisons among the five programs 

* " ' * • ' * 

(some cutting across as many as 11 schools). The second set required 
• »>. ** * * ■ 
comparisons of the. four schools ihto v;hicH the Junior High Program 

■’’ '■ \’- 

was introduced. Tlie .third, also a comparative schoc^l analysis, in- 
volved six schools having .the- Early Primary Program. \The tv;o ^school,' '<• 

\ 

analyses v/ere pbssilole because tv;o. programs contained large concent- 
rations .of parapro'fessionals in many schools. Therefore, v;hen. the - 

, ■ ' • ■ V N 

original stratified sample of pa.raprofessionals v;as drawn for the , 
progrcim comparisons, enough paraprofessionals (end their teachers) 

* V ^ 

v;ere included in these schools .to develop an adequate summary school 

# 

score for each of the five independent, variables and the dependent 
vanieble; The procedures -that follow v;ere used to obtain the summary 

' • I ’ 

scores. - • 



De cree- of Im pl ement at ion -. Fifteen items about the paraprofesr 

sionals' role performance v;ere included on the IMQ in order to assess 

.1 

the degree of actual implementation. The "items emerged from our 
analysis of the paraprofessionals' responses on the PDO, prog.ram 
documents discussing the nature of the innovation, end our profes- 
sional judgments.* Ten of the items used for the Early Primary Program^ 
for example, described behavioi; that pciraprofessionals in this progr.am 
would .be exhibiting if they were, properly carrying out their role 
and five activities that they \iou Id not exhibit , if they were properly 
implementing the role. The "positive" dimenSiohs v;ere: 
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1) work 'v/ith cl^ildren in cl?.ssrooms on reading skills. 

4 

7) work v;ith children in classrocxns on reading '.comprehension, 

3) v/orlc v;ith children in classrooms on math skills. 
t4) work v;ith children v;ho need remedial work in reading. 

5) work primarily v»ith small groups of children on school subjects. 

' I « ■ * 

7) plan' classroom activities v;ith the teacher (s), 

9) perform clerical duties A/ithin the classroom such as taking 

attendance, correcting papers, preparing and duplicating materials, 
ID confer v;ith teachers about children. \ 

•j 5) attend training sessions for paraprofessionals, 

] 

15) keep abreast of materials on education that v;ill be helpful on 
the job. 

The five inappropriate activities v;ere : 

6) v;ork primarily on Don-instructional matters with children in 

. the classroom. 

■8) act as a substitute v;hen a teacher is absent from school. 

10) help to prepare snacks and lunches for childrerf. 

11) help in the health clinic to care for those children in need 
of medical attention. 

13) visit children and their parents at home regarding instructional 
• : e.nd/cr non~instructional matters. 

The teachers v:ere asked, as observers, to estimate hov; frequently 

tiieir paraprofessionals v;ere engaging in each of tiiese 15 activities 

. ■ ■ • \ 

V 

\ 

using the follw;ing scale: .i.=never, 2=seldom,. 3= occasionally, 

V. ' ■ ' ' ‘ ’ 

4= usually, 5=alv;ays, N=cannot judge. The paraprpfessionals were 
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asked to assess otlier pa rsprolessioric Is, witli : v;hom they had contact, 
on these 15 activities. T’eighting the items equally, the responses 
of each teacher and par rpr of ess ion el vere suituned (after reversing 
the scoring of the negative items) and divided by 15, The resulting 
mean scores represented the subjects' overall assessments of the degree 
to v.’hich- paraprofessionals were carrying out the role. Subjects' mean 
scores were then averaged either across programs or schools (depending 
upon the- analysis) , These mean scores, in turn, represented summary 
indices of implementation for programs or schools, , . 

Employing different sets of items, this procedure v;as followed 
to obtain summary scores for programs and schools on foUr of the 
independent variables: the capability of paraprofessionals, their — 

willingness, the availability of necessary resources, and organizational 
support. Since these variables are specific for the innovation being 
studied, establishing the validity of the instrument is especially 
difficult. One v;ay of providing evidence on the validity of the measure, 
although not without apparent limitations, v;ould be to see if the results 
of distinct groups using the instrument are similar. 

For each program the mean scores of teachers, paraprofessionals, 
and some administrators (v;ho were given the IMQ for other purposes) on 
the five conditions vjere separated and subjected to analyses of vari- 
ance to determine whether assessments of these conditions differed among 
the "three groups. The results are summarized in Table 3, Only two of 
' the P5 AlJOVA's indicated significant mean differences between teachers 

I 

/ 

and paraprofessionals: the /.NOVA on Early Primary resources and the ' 

i>.N0VJ. on Intermediate willingness. The similarity of results obtained 
from the tl^ee groups measured independently using the same instrument. 
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adds confidence to the validity of our measures. Once these results 
were obtained, it became apparent that the inclusion of the data from 
the administrators introduced an unnecessary encumbrance on the research 
analysis. Therefore, they v;ere omitted at this point, and the scores 

of teachers and ibaraprofcssionals were pooled. The scores of the 

■ \ 

teachers and paraprofessionals were pooled in order to arrive at the 
final summary scores to |be used in the program and school analyses. 

The summary mean scores for the five programs are reported in Table 4, 
for six, of the schools in the Early Primary Program in Table 5, and 



for the' four schools in the Junior High Program in Table 6. 

The summary program scores on implementation as evidenced, in 
Tab^ 4 ranged from 4.04 to ,.3.S5. On a scale from 1 (low) to 5. (high), 
is can be interpreted to mean that overall the five programs were 
being implemented at a fairly high level and that the variation betv/een 
them was not great. The summary school scores for the six Early Pri- 
mary schools (Table 5) ranged from 4,19 to 3,60 and in the Junior High 
schools (Table 6) from 4.17 to 3.52. 



Pc- r a professional Ability t o ' Carry Out the Role ; To obtain a measure 



of the degree of paraprofessional ability in each program, only the 
"positive" items used in meas\jring implementation were employed. For 
example, in the Early Primary Program respondents were asked to judge 
hcwtf capable were the paraprofessionals to perform the foliating 
activities : ^ 

1) \;orking v;ith children in classrooms 'on reading skills. 

I • , 

2) v7orking v;ith children in classrooms on reading comprehension, 

3) working i^ith children in classrooms on math skills, 

■■■■ 12 ' 
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<.) v;orking with children v/ho need remedial work in rending, 

5) working primarily v/ith small groups of children on schccl 
subjects, 

6) planning classroom activities with the teacher (s). 

7) performing clerical duties within the classroom such as taking 
attendance, correcting papers, preparing and duplicating materials 

8) conferring v;ith teachers uoout children. 

9) attending training sessions for paraprofessionals. 

10) keeping abi'east of materials on education that vjill be helpful 
on the job. 

They v;ere asked to use the following code in responding: l=completely 

% 

incapable, ?=somewhat incapable of, 3=scxnevhat capable of, 4=very capable 

of, 5=ccmpletely capable of, and N?=No basis for judging. The summary 

scores for programs and schools reported in Tables 8, 9, 10 were computed 

in the manner employed in measuring implementation. Variation in ability 

by program ranged from 4.54 to 3.47. Variation in ability by school in 

the Early Primary Program ranged from, 4. 47 to 2.91 and in the Junior 

High Program - from 3,76 to 2,51, On a five~point scale' from 1 (lo\\’) to 

5 (high) the variation between schools within the two programs v;as much 

greater than across projects, even though ability was still generally 

♦ 

high across programs. 

Willingness of Paraprofessionals ; To assess the degree to which 
paraprofessionals in each of the programs v;ere willing to implement 
their role, the positive items, as used in the ability section, were 
presented with the following code: l=never willing to, 2=seldom willing 

to, 3=occasionally willing' to, 4=usually willing to, 5=always willing to. 
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and N=no basis for judcjing. Variation in willingnc.?s across programs 
ranged fron 4.70 to 4.02 (Table 8)'. In the Efr’.y 3?.:lmary scnools the 
variation v/ant from 4,89 to 4.12 (Table 9) and in the Junior High 
schools from 4,54 to '3,96 (Table 10), Vlillir.gnsss also apoaored to 
be generally high. 



•■a- 



Avai la bil it y of No oesggr .y Reso urce s a nd M o!: oria ls ; To measure the 
adequacy of resources and materials resijondents were asked to judge the 
adequacy of a) materials, b) physical facilities, c) financial resources, 
and d) staff, on a fiv 3 ”point scale from l=inadequate , 2=somewhat inade- 
quate, 3=somev;hat adequate, 4=adequate, 5=completely adequate. Each 
subject's, mean score based on these four items was . calculated and then 
similar calculations were made for programs and schools. Compared to 
ability and v;i].lingner?s, the availability of resources across programs 
and schools was lower. Mean program scores ranged from 3,30 to 2,47 
.(Table 8) , in the . Early . Priiuary program by school from 3,40 to 1,43 . 
(Table 9), and in the Junior High schools from 2,72 to X,96 (Table 10), 
Organizati o nal Cc =m oatih:.7,itv ; Seven sources of organisational 
incompatibility were identified for this study and put into question 
form. Subjects were asked to judge to \:hat extent standard procedures 
or, policies in schools were interfering with the installation of the new 
paraprofessional role. Given the following code — l=no ejid 2=yes — 
they were asked to answer the folloiving questions for the paraprofessionals 

1) Do expectations or policies of their school administrators 

interfere v;ith their carrying out the role?’ • ' 

2) Do expectations or policies, of their teachers interfere 
v;ith their carrying out the role? 
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3) Do doncuds- ia?.de ,of them by students qet- in the v* 3 y? 

4) Co c.; peronts interfere? 

5) Co -,V;- o‘- the ori.vcnvinity iiicevlere? 

6) hre thr.ro a::v dir.V.r:- r:t policies r 'risols that act 

f*s o)? iicie'.j to their carrying cut tba roloV 

7) Are e:cpsctatio*i3 of personnel in the program ’vhich 

iu'cerfe.re v;ivii their effective prxfer.v nnee ? 

A sumn.ary score for each subject based on thasc itams was corri''uted , 
Scores cculd rar/.s from .1.00 (coir.palibility) to ?.00 (incaT;:.-,;;tibiIity) . 
Prcg?.v-jm sc ?ros r-.nyed fvoiTi 1.00 to 1.03 (Table a). Early P-r.imary school 
scores var.xod fror-n 1,00 to 1.13 (T?..ble 9), and Junior High school scores 
froiTi 1.00 to 1.14 (Tabj.e 10), Uith high CiTTi’pat'.i.b.ility as 1.00^ thore ' 
was almost no v;rJ7ation either by program or by schoo.ls within prerrams. 
High compatibility appeared to be present. 

on t ha IHf • ; The results may be 

concaminatti'j by tha f.act that sjjbjects were automatic?..' jy judging C;very~ 
thing as hrgh" or as '.'.ow''. To detHrmine the .iu-deperiannce c;f the respon* 
dents* judg-nents on the five variables discuss^;*! above, the.ir raean scores 

I 

were correj.a ted (Tab.1.e 7). The inte.*:corx*-i:>,ticrs were comput-ad for each 
prog.ram. Although many of the corre-lations betwaon i:.\t;jlemer.tation and 
the foux inJnpendent variables were significant, in or.iy two (both in the 
same prcgJcain) is more rhun 25 per cent of the variance in i:.\T;.iementation 
explained. The strong&r;t corjcistent corralatib.n among the independent 
variables was that between rat.ings on ability and on willingness, e.g., 
in the Preprimary Program the correlation was .92, the Early Primary .83, 
Late Primary .89, Intermediate .31 and Juiiior H.igh School .68. The 
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weakest consistent correlations were between ability and compatibility 
and between resources and canpatibility , The findings sugg.est that the 
respondents' ratings on the variables v;ere/ in the majority of instances, 

f 

substantially independent of each other, 

clarity of the Paraprofessibnal Role ; The degree to which the 

f 

/ 

paraprofessionals were clear abq'ut the innovation and their role was 

I 

I 

determined in the' following manjj^er. It was reasoned that the degree 

■ I 

of consensus of the paraprofessionals on the 56-item RDol would reflect 

f ; 

c 
( 

the degree of clarity prevail|ng in the programs and schools. Clearly, 

r ' . , 

/ 

if there was little agreement' found among them about how much time they 

should spend on these activities, this would be evidence of lack of 

/ 

consensus; the lower the consensus, the ' lower the clarity. 

It was judged that there was high paraprofessional agreement on the 
importance of any given activity when at least 80 per 'cent of the para- 
professionals.' responses fell within no more than two categories (e.g., 

4's and ^'s or I's and 2's). The responses for each item were: 1= a 

person with this job should be spending none of one's time, 2= very little 
of one's time, 3= some. of one's time, 4= a good deal of one's time> and 
5= as much of one's timers possible. Among the Preprimary paraproc;*^ ^ 
fessionals there v;as agreement on only 7 per cent of the iterr.c,-' among 
the Early Primary subjects 5 per cent, among the Late Fcimary respondents 
9 per cent, among the Intermediate 29- per cent, and among the Junior High 

paraprofessionals 7 per cent (Table 8)", The evidence suggests that there 

^ «• . . . • 

was a great lack of consensus, and thus lack of clarity, in each program 
and fvorther, that there was little variation betv;een programs. Variation 
was greater within the Early Primary schools ' ranging from 34 per cent to 
7 per cent agreement, and v;ithin the Junior High schools from agreement 
on 52 per cent of the items to 7 per cent. 
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Rss ults 

On the basis of the summary scores computed above, the programs 
(and schools) were rated as high, 'moderately high, moderately low, or 
lo\-; on each of the five conditions depending upon which quartile of the 
scale for that variable the svonmary score fell. Then, composites of 
ratings for the programs (and schools) were formed and on the basis of ■ 
these composites, the programs were ranlced on implementation. For example, 
a program with thrca "high" ratings, one "moderately high" rati.ng, and one 
"low" rating was ranked above a pregram v/ith two "high" ratings, one 
"moderately high" rating, one "moderately low" rating, and one "lov;" rating 
This predicted order, made independently of the assessed rank order based 
on the implementation index, was computed and compared with the assessed 
implementation index. A Spearman's rho was calculated, and its level of 
significance specified. Identical procedures v/ere employed for the two 
school analyses. If, as predicted on the basis of the theory, the ttoee 
analyses uncovered strong rank order correlations between the predicted 
rankings and the assessed rankings of schools or programs. on implementation 
this would be interpreted as support for the theoretical model. 

Table 11 summarizes the program analysis, which, according to the 
theory, should reveal a strong association between the ^fedicted rank 

I 

ordering of the programs on implementation and the escp-'jsed rank ordering 
of the programs on implementation, ?s a result of the composite ratings, 
the Preprimary Program v;as ranked as number one. Early Primary as second. 
Intermediate as third, and the Late Primary and Junior High Programs as 
tied for 4th and 5th places. -The rank order based on -he i.i'.plementation 
index also placed Preprimary one. Early Primary two,, Junior High three, 
and Late Primary and Intermediate as, tied for 4th and 5th places. The 



• • r 



correlation of ,778 betv;een these tv/o rank orders was in the expected 
direction and significant at the ,08 level, " 

The rank order analysis of six of the schools in the Early Primary 
Program is summarized in Table 12, The association between the pre- 
dicted order of the schools on implementation and their assessed order 
is strong and in the expected direction, h Spearman's rho was calculated , 
to be .872 and was significant at the ,02 ^level, 

i ■ • 

ihe results of the rank order analysis of the four schools having 
Junior High Paraprofessional Programs are summarized in Table 13, The 
relation between the predicted and assessed rankings is again strong and 
in the expected direction, A Spearman's rho calculated to be ,995 was 
significant at the .05 level. 

In sum, the three analyses revealed strong associations and acceptable 
levels of significance between the expected and measured rank orders 
despite the size of correlations required with few cases. 

Discussion and Implication s 

Hie theoretical model of implementation under' examination specified 
that the extent to which organizational innovations are implemented 
depends upon the relative presence of five conditions : tlriL- clarity that 

organizational members, who, must carry out the change,, have about their 
role, their ability to make the necessary efforts, their willingness to do 
so, the availability of the necessary resources and and the 

■ t ■ 

carpatibility and support of the existing organization^ It was argued 
that 'this theoretical model of implementation would bs^ supported by the 
analysis if the programs or schools that ranked higgler on implementation 
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on the basis of the five independent variables would also rank higher 

■T 

on assessed implementation. Since strong correlations were found 

\ 

between predicted and assessed rank orders in each of three analyses, 
the hypothesis v;as confirmed. 

It v;as also argued that according to the model, schools or programs 

high on all five factors would demonstrate high absolute levels of 

. . . ' ' ' 
implementation. Ihis, too, was confirmed by- the investigation. None of 

the programs or schools were high on all five conditions, and there were 

no examples of thorough implementation as evidenced by the absence of 

mean scores between 4,75 and 5.00. Close inspection of Tables 11-13 

reveals that the summary implementation scores range from' 3.52 to 4. -19 

and that nearly all programs end schools had three or more ratings of 

high or moderately high, while receiving low ratings on role clarity. 

Since the typical composite ratings, according to the model, would be 

predictive of substantial^ though not complete, implementation, the 

theory v;as also supported in this manner. Therefore, these findings 

lead to the conclusion that the model has utility in e:^laining the 

implementation of organizational innovations. 



Implications for Future Research ; This study sheds light on 
conditions that accounted for variations in implementation of innovations 

f * 

Further replication studies in settings where similar paraprofessional 
programs are introduced are needed. Moreover, to test the usefullness 
of the model, its utility in .educational settings with innovations other 
than paraprofessionals. are necessary, as well as settings cutside of 
education. 
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Although this study isolated a number of antecedent conditions 

for implementation, it did not account for why they prevailed to the 

• \ 

extent that they did, e.g., the lack of role clarity among parapro- 

fessionals and high level of their willingness to perform their role. 
Research along these lines is important. 

Other research must be directed to refinements .in measurement, 
especially in the areas. of reliability and validity. Though some 
exploratory work v;as done on reliability, the consistency results were 
not as strong as anticipated by the research team, A. revision of the 
RDQ taking into consideration some of the .possible limitations . listed 
earlier would be desirable. Reliability of the. RDQ might be assessed 
through correlation of responses' betv;een two randomly selected groups 
or a test-retest design with the same subjects. - Similar reliability 

^ i 

indices should be explored for the IMQ, in this case two randomly 
selected groups may be. the preferred method due to the length of the 
questionnaire and the demands on the respondents' time'. Though 
exploratory work on the validity of the measures in the IMQ 'was done, 
much more work is required, 

/ . i ^ 

Although^ model yielded high predictability when compared 

to the imRl^entation index, the variability of both the independent 






variables and implementation was restricted. Two plausible explanations 

I 

are that the conditions and programs were being implemented to a similar 
degree or that the methodology may have contributed to the limited 
variance, The latter deserves further exploration. In the present study 

^ A 

items for ^implementation, *. capability and v>illingness v;ere contingent, 
in part, upon role definitions provided by paraprofessionals , By asking 
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them to define their own roles, there may have been a tendency to 
define it on the basis of. what they were actually doing, thereby inflating 
the degree of assessed implementation. ;.n alternative means of deter- 
' ^mining the parameters of^a given role might be to administer the.RDQ 
to a group familiar with the goals of the program, but who are' not them- 
selves involved in the role. ^Wo possible groups might be the trainers 

' -■s ' 

of the paraprofessionals and the Title I District Administration staff. 
Research addressed to some of these concerns is- currently in px'ogress. 



Evaluations and Social Science Research; This study was confined ’ 
to the analysis of one area of social science, i^e.<, the implementation 

• ^ ‘ ^ . ii 

of organizational change. It is apparent that evaluation studies can be- 

, . .1 st 

used to e:^lore other areas and to test other social science; theories 
relev’ant to education. The illustrations that follow are indicative of 
some of the kiii.ds of research that might be pursued. . . 

Although little systematic research has been done on the relationship 
between staff ethnicity, and racial composition and studnnt identification 
and achievement, the claim continues to be made, as several books on 
decentralization attest, that students v;ho cai^^dentify with the ethnicity 
of their teachers will perform well end develop positive self images. 
Similarly, the claim is ^^ten made that parental participation ..in school 
decisions affects the' learning of studen'tS. Do students with, the same 
ethnicity as their teachers de'velop positive self images? Do the 



children whose mothers are paraprofessionals achieve more than children 



\ 



whose parents^ are not involved, in school affairs? Implicit in questions 
such as these are psychological theories that warrant further testing. \ 



I 






-V 



A 



22 . 



Others, notably the United Federation of Teachers, have maintained 
that smaller classes will” improve student achievement, especially in the 
area of reading. Implicit in this argument are sociological and psycho- 

I . ' . 

logical theories about the teaching and learning of reading. Evaluations 
o£ reading programs v;ould be appropriate \vays to test theories abopt the 
reading process, 

* Similarly, the extent to which political, socialization accompanies 

parent participation in funded programs is an aspect qf educational 
evaluations that often goes unnoticed. -Schools are socializing agents 

for students, teachers and parents. And that .fact is borne out most clearly 

. < * ' 

in ancillary ^programs designed to complement the conventional school pro- 
grams. When -community decisions are made regarding the selection of 
teaching materials the >^fcomponents 'of socialization are most evident. Yet 
• all too of ten revaluators assess the extent of resources, not how they are 

4, • . j 

selected. " " . 

'.Economic matters are also subject to. systematic analysis- in an educa^ 
tional setting. • li congressional debates are any -index, ,one could e,asily' 
get the impression the allocation of federsl funds to local- school districts 
is the critical determinant in predicting student achievement. But is 

“ * i. 

this true? - And if not, what influence do additional funds have on student 
performance? The issue is clearly stated as "Whet is the return on in- 
vestment?" Another related issue is whether spending in an institution 

/l 

other than the school is likely to reap greater student benefits. For 
example, are achievement levels likely to improve more significantly if a 
maintenance allowance were given to the family rather than the school? 

Questions, such as these, have generally net been studied., by educa- 
tional evaluators. Yet, large scale evaluation studies, particularly those 
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1 ' 

involving Title I, Title III, and State Urban Education funds, are . 
vehicles for this pvirpose. What is needed is a clearer recognition, 
than has heretofore been realized, of the potential for social science 

I . . ‘ 

' / 

research in these evaluations and xts importance for education. 

/ -• 
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Percent of Sample P»eturns from Five Paraprofessional Programs 





Role Definition 


Implementation 




Questionnaire® 


Questionnaire^ 


Program 


Sample Size % Return 


Semple Size^ % Retmrn 




(N) 


(N) 



Preprimary 

t 


13 


100% 


(13) 


11 


100% 


(11) 


Early Primary 


62 


97% 


(60) 


95 


74% 


(70) 


Late .Primary 


- ‘ '13 


oD*o 


(11) 


16 


80% 


(14) 


Intermediate 


12 


100% 


(12) 


17 


94% 


(16) 


Junior High 


27 


96% 


(26) 


42 


79% 


(33) 



cl 

Paraprofessionals only 

^^Paraprofessionals and Teachers 

% 

r c 

The expected sample size of the implementation questionnaire should - 
have been double that of the role definition instrument. Because 
some paraprofessionals were transferred from one program to another 
during the year, the implementation sample size reflects this attri- 
tion. 
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TABIE 3 



F ratios end Significance levels for Analyses of‘ Variance of 
the Responses among Teachers, Paraprofessionals, and .Administrators® 

in Each Program on Five Conditions 



Level .of 



Program ■ 


Variable 


F ratio (d f) 


Significance 


Preprimary 


Implementation 


0.4094 


n.s. 




Ability 


1.2179 


n«s. 


r 


v:illingness 


0.5303 


n.s. 




Resources 


1.3313 


n.s. 


♦ I • 


Compatibility ' 


. 1.0497 


n.s. 



Early Primary 


Implementat ion 


1.4501 


(2 


& 


70) 


n.s. 




Ability 


1.9790 








n.s. 




V7illingness 


0.0224 








n.s. 




Resources 


4.6957 








<.05 


• ' 


Compatibility 


0.4977 


• 








Late Primary 


Implement a t i on 


■ 0.3938 


(2 


& 


15) 


n“.s. 




Ability 


“ 0.0747 






' 


n.s. 




Willingness 


0.6991 








n.s; 




Resources 


0.4066 








n.s. 


• 


Compatibility 


0.1454 








n.s. 


Intermediate 


Implementat ion 


0.5'978 


(2 


& 


20) 


n.s. 




Abi.i.ity 


0.6024 








n.s. 




. Willingness 


4.3903 








C-05 




Resources 


1.7391 








' n s . 




Co.->ipatibi lity 


1.0377 








n.$. 


1 

Junior High 


Implementat ion 


* 0.3127 


(2 




55) 


n»s. 




Ability 


O'. 32 33 








noS. 




Willingness 


0.0419 








n.s. 




Resources 


0.5909 








n.s. 




' Compat ibi li ty 


,0.0464 








n.s. 



For the purposes of future study, the responses of various administrators 

V. 

in each program were included in this analysis. 
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TABIE 4 



CrCTucry Firojoct Scores c.nd Ctfnc.rrd Deviations 
For Implementation and Four Iiidependent variables 



Program 




Independent 


Variables 






Ability 


^ vmiing- 
ness 


Resources Corapat” 
, , ibility 


Implemen- 

tation 


Preprimary X 
SD 

■ N = 11 

j 


4.5364 

0.5697 


• 4.7000 
0.5848 


3.2955 

0.9926 


1.0000 

0.0000 


4,0301 

0.3371 


Early Primary X 
SD 

N = 70 


4.0908 

0.9130 


4.4321 

0.8932 


2.6993 

1.3332 


1,0247 . 

0.0190 


3.9673 

0.5730 


Late Primary X 
SD , 

N = 14 


3, -7009 
1.2406 


4.0241 ' 
1.3712 


. 2.6875 
1.5'230 


1.0000 

0.0000 

7 


3.664 3 
, 1.1S81 ■ 


Intermediate X 
SD 

N = 16 


3.6328 

0.5789 


4.4012 . 
0.6069 


2 -.5209 
1.3009 


1.0060 

0.0040 


3.6517 
0.31 .:>ii 


Junior High It 
SD 


3.4728 

1.2930 


4.1406 

1.2211 


2.4722 

1.2445 


1.0328 
' ’0.0253 


3.8147 

0i0G07 



N = 33 
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TABLE 5 






Sur.*.mcry CcTxci Scores and Stand crd Deviations for 
Implementation of the- Early Primary Program and Four Independent 

Variables 



School Independent Variables 





Ability 


Willing- 

ness 


P.esources 


Compat- 
' ibility 


-Implemen- 

tation 


A X 


4.4666 


4.6534 


3.4000 


1.1286 


4.1886 


SD 


0.6405 


0.3699 


0.9117 


0.0217 


0.2472 


N = 5 








• 




B X 


2.9089 


4.6032 


1.7583 


1.0429 


3.6554 


SD 


0.6168 


0.3699 


1.2841 


0.0097 


1.2349 


N = 10 


• 




- 






C X 


4.6111 


.4.8889 


3.8056 


1.0000 


3.9988 


SD . 


0.3180 


0.1364 


0.7265 ' 


0.0000 


0.2968 


N = 9 








-• 




D X 


4.3800 


4.4000 


1.8500 


1.0858 


3.9190 


SD 


vOoCOSe 


0.5891 , 


0.8768 


0.0128 


0.3244 


N = 5 








. 




E X 


2.9918 


4 . 1205 


3.1591 


1.0702 


3.60li 


SD 


lc4384 


1.4646 


0.5157 


0.0082, 


0.3732 


N = 11 


i\ 




* 






F ■'K 


4.2300 


4.6611 


1.4286 ! 


1.0000 • 


4.0291 


- SD 


0.2419 


0.3131 


1.3973 


0.0000 


0.1705 



N = 7 






30 




TABLE 6 



SxjmuAary School Scores and Standard Deviations for Imple- 



mentation of the Junior High Prograan and Four Independent Variables 



School 






Independent Variables - 


Ability 


Willing- 

ness 


Resources 


■ Compati- 
bility 


Implementation 
of Junior High 
Program 


A 


Y 


3.7636' 


3.9633 


2.6429 


1.0119 • 


• 

- 3.5373 




SD 


0.6211 


0.7858 


1.3374 


0.0101 


0.3624 


N= 


7 












B, 


X 


3.5472 


4.2247 ■ 


2.5075 


1.1346 


4.isi4 




SD 


1.2875 


1.4483- 


1.3025 


■' 0.0494 


0,6922 


N= 


11 








■■ 




C 


X 


3.9834 


4.5431 


2.7188 


1.0000 


4,1711 




SD 


0.5334 


0.4219 


0.7372 


0.0000 


0.6065 ■ 


N= 














D 


T 


2.5054 


3.7246 


1.9643 • 


1.0000 


3.5176 - ' 




SD 


1.9717 


1.7789 


1.6100 


0.0000 


1.2808 ' 


- N= 


7 


1 


r 




f 





\ 






• 






< 






- 


s 






— • — r 


-■ 




• • 






; 


• 


b « ' • 

^ 1 


1 




/ 














» -.S 










^ ’ .•'tj • . 




. 




TABIE 7 


•“ * -V 

t Five Conditions as . ^ 




. Independence of 


a 

Staff Judguvents abov. 


t 


Measured by the 


Intercorrelations of 


their Surmaary. Scores • 




' 




Variable ' • ' .* 




Program 


' Implemen- 




Willing- ' _ , N 




• 


tation 


Ability 


ness ’ Resotirc6s 




‘ 


(1) 


(2) 


(3) (4) •-> 




Pr epr imary (N= 15 ) 






--y V : 

. , -i 




2, Ability 


.530* 








3. Willingness 


.502* 


.916* 






4, Resources 


.382 


.419 


.439 




5, Compatibility 


^072 


-.375 . 


-.298 .224. 




Early Primary (N=73) 






* •# 




2. Ability 


.475* 




• ^ 

. ^ 1 




3, Willingness 


.495* 


.333** 


t 




4, Resources 


.212 


.084 . 


.-.065 




5, Compatibility 


-.107 

• \ 


-.074 


-.164 .187 

• * . * 




Late Primary (N=20) 




• 


4 - . . » 




2. Ability 


.791** 








3. Willingness 


.829** 


i0S8**.‘ 


. ■ 




4, Resources 


.466* ^ 


.570**- 


.443* 




• 5, Ccanpatibility 


.396* ' 


.493* • 


.364 . ''.536* . « 




Intermediate (N=24) 










2. Ability 


.349 


■ 




- 


3. Willingness 


• .385 


.305 






4, Resources • 


\ .152 


.288 


.393*' 




5, Corapatibility 


-.085 


-.074 


.228' .1"34 




Junior High (N=37) 




• 














2. Ability . .. 


.403* 








3. Willingness 


.469'*^ • • 


.680** 




4, Resources 


.096 


*252 


.474** ' : • ^ ^ 


' 


5, Cranpatibility 


-.042 


■ .101- 


..'.201 % .'.009 




* p <^.05 




e 

4 ; • 






p /.Ol 






J . 
$ 




a 










Includes Teachers 


, Administrators, and 


Paraprofessionals 

s'* ■ • 
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